Head-mounted displays having curved lens arrays and generating elemental images for displaying

ABSTRACT

An example apparatus for displaying stereo elemental images includes two coupled eyepieces. Each of the two eyepieces also includes a curved screen to display a number of elemental images. Each of the two eyepieces also includes a curved lens array concentrically displaced in front of the curved screen to magnify the elemental images. Each of the number of elemental images is magnified by a different lens in the curved lens array.

BACKGROUND

Head-mounted displays (HMDs) are used to present virtual reality scenes.For example, HMDs may display a pair of images rendered for each eyethat may be refreshed with movement of the head to present users with athree-dimensional virtual environment.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an example head-mounted display thatincludes a curved lens array;

FIG. 2 is a diagram illustrating an example curved virtual realitydisplay for one eye;

FIG. 3A is a diagram illustrating an example viewing zone for a singlelens;

FIG. 3B is a diagram illustrating an example compound viewing zone forthree lenses;

FIG. 4 is a diagram illustrating an example compound viewing zone for alarger number of lenses;

FIG. 5 is a diagram illustrating an example thick lens model;

FIG. 6A is a diagram illustrating an example lens design using asphericsurfaces;

FIG. 6B is a diagram illustrating an example lens design using free-formsurfaces;

FIG. 7A is a diagram illustrating an example heterogeneous free-formlenslet array;

FIG. 7B is a diagram illustrating an example 6 lens heterogeneousfree-form lenslet array adapted for eye rotation;

FIG. 8 is a diagram illustrating an example system for rendering 3Dcontent to a virtual plane;

FIG. 9 is a diagram illustrating an example system for rendering 3Dcontent to a cylindrical rendering surface;

FIG. 10 is a diagram illustrating an example mapping of an image planeonto an elemental image;

FIG. 11 is a diagram illustrating an example range of locations for acenter of projection when rendering elemental images;

FIG. 12 is a block diagram illustrating an example method of renderingelemental images using ray tracing;

FIG. 13 is a diagram comparing example spot diagrams of field pointsaligned with a fovea of a rotated eye for identical and heterogeneouslens designs;

FIG. 14 is a flow chart illustrating a method for generating elementalimages to be presented via a head mounted display;

FIG. 15 is block diagram illustrating an example computing device thatcan render and present elemental images via a head mounted display; and

FIG. 16 is a block diagram showing computer readable media that storecode for rendering elemental images to be presented via a head mounteddisplay.

The same numbers are used throughout the disclosure and the figures toreference like components and features. Numbers in the 100 series referto features originally found in FIG. 1; numbers in the 200 series referto features originally found in FIG. 2; and so on.

DESCRIPTION OF THE EMBODIMENTS

As discussed above, Head-mounted displays (HMDs) can be used to presentvirtual reality scenes. However, due to the limitations of conventionaloptics such HMDs may be bulky in design and may exhibit poor visualperformance towards the periphery of their displays: the effectiveresolution degrades, geometric distortions and color aberrations becomemore prominent. In addition the effective FOV of the conventional HMD islimited by the optics.

The present disclosure relates generally to idea to an improved HMDusing curved heterogeneous or homogeneous lens array as the opticalelement, and techniques for rendering and presenting virtual realityimages for the improved HMD. Specifically, the techniques describedherein include the use of HMDs with curved lens arrays, and anapparatus, method and system for rendering and presenting virtualreality images via the HMDs with curved lens arrays. An exampleapparatus includes two coupled eyepieces. For example, the coupledeyepieces may be a left and right point of view for each eye of a user.The apparatus may be a head mounted display. Each of the eyepiecesincludes a curved screen to display a plurality of elemental images. Asused herein, elemental images are subsets of images displayed via ascreen of each eye piece in an HMD. Each of the eyepieces also includesa curved lens array concentrically displaced in front of the curvedscreen to magnify the elemental images. Each of the plurality ofelemental images is magnified by a different lens in the curved lensarray.

In addition, the present disclosure includes a method for renderingvirtual reality images, including receiving an image to be presented anda virtual distance from the eyes of a viewer. The method also includesrendering a stereo view of the image for each of the eyes at a virtualsurface located at the virtual distance. The method further includesmapping pixels for each stereo view from the virtual surface toelemental images of a per-eye display using a per-lens projection model.The method includes pre-warping the elemental images based on a per-lensdistortion model to compensate for a lens distortion. The method alsofurther includes sending pre-warped elemental images to a head mounteddisplay to be displayed.

Using the curved lens array and techniques described herein thus enablea thinner form factor for virtual reality HMDs as compared to currentlyflat screen HMDs with conventional optics. In typical design, a flatOLED/LCD display is positioned behind the optical element, at somedistance slightly less than its focal length. This generates a virtualimage appearing at a distance much farther than the actual display. Forexample, VR HMDs may be inherently bulky due to the long optical pathsrequired by traditional refractive optic design acting as magnifyinglenses. This bulkiness may only worsen as field-of-view (FOV) increases.To roughly measure how bulky a VR headset must be, one can simplycalculate a ‘total thickness’ at any point from the user's face. Forexample, the total thickness may be the sum of the face-to-lensdistance, the lens thickness, and the spacing from lens-to-display. Formany systems with decently large lenses for FOV, this thickness can beup to 100 mm or more, not including housing and electronics. Thetechniques described herein overcome the long optical path required oftraditional VR optics by replacing the bulky main lens of relativelylong focal length with an array of smaller lenses of much smaller focallength. Moreover, using curved lens array and techniques describedherein enable a very large (>180 degree) FOV and a dramatically thinnerdesign contoured to the human face, while maintaining acceptableresolution. In addition, the techniques described herein can useinjection-moldable polymer optics and commodity-cost OLED panels,thereby reducing complexity and cost. Moreover, the method for renderingvirtual reality images may enable elemental images to be computedquickly and accurately in real-time low latency applications. Forexample, it can be implemented as a post-processing mapping of pixelsrendered using conventional 3D graphics methods from planar surfaces toelemental images, thus adding a very little cost to the rendering costfor conventional stereo HMD. Such a mapping can use a plurality ofmethods, such as precomputed look-up tables for various eye conditions,or computing inexpensive mapping function comprising of superposition ofprojective transformation and distortion pre-compensation warping, orcombination of both. In some examples, eye pupil tracking may alsoenable real-time measurements of eye parameters, hence, real-timecorrection of displayed elemental images so that virtual rendered imagesremain fused and sharp at any eye condition.

FIG. 1 is a diagram illustrating an example head-mounted display thatincludes a curved lens array. The example head-mounted display isreferred to generally by the reference number 100 and can be implementedin the computing device 1600 below in FIG. 16 using the method 1500 ofFIG. 15 below.

The example head-mounted display 100 is shown with respect to two eyesof an example user, including a left eye 102A and a right eye 102B. Thehead-mounted display 100 includes two separate curved screens 106A,106B, associated with the left eye 102A and right eye 102B,respectively. The curved screens 106A, 106B are associated with curvedlens arrays, 104A and 104B, respectively. An inter-pupillary distance isindicated by an arrow 108 between pupillary axes 110A and 110B. Forexample, the inter-pupillary distance may be approximately 2.78 inchesor 70.61 millimeters.

As shown in FIG. 1, the head-mounted display 100 may present a pair ofvirtual reality images or video to the eyes 102A, 102B of a user. Forexample, the VR images or video can be rendered for presentation on thehead-mounted display 100 as discussed below. As can be seen in FIG. 1,when viewed together through a head mounted display, the curved screens106A and 106B can present an image with a view angle of 180 degrees.Thus, a high quality image may be presented to the periphery of eyes102A, 102B. Such a high quality image provided to the peripheral visionmay improve user experience by making virtual reality images appear morerealistic to the eyes 102A, 102B.

In some examples, each of the curved screens 106A and 106B may displayindividual elemental images as described in greater detail with respectto FIG. 2 below. Likewise, each of the curved lens arrays 104A and 104Bmay be composed of any number of lenses. For example, each individualelemental image may be associated with a particular lens of the lensarrays 104A and 104B. The curved lens arrays 104A and 104B can magnifythe individual elemental images to be projected into eyes 102A and 102B.Thus, a smaller focal length than traditional HMDs can be used, reducingbulk and weight of the HMD. Therefore, there are some distinctadvantages to curving the lens array, either in part or entirely, foreach eye. In some example, using curved lens arrays may result inimproved off-axis performance. For example, for a cylindrical curve, auser may be approximately viewing all central lenses across thehorizontal direction as an on-axis lens. This may provide an ultra-wideFOV with very sharp imaging along the entire curvature and not justalong the central axis. For example, the FOV may be more than 180degrees. In some examples, a freeform lens refinement may be used forthe vertical dimension. For example, lenses in the vertical dimensioncan be modified with shapes based on their location to improvesharpness. In some examples, the curved lens array and the curved screenmay include a spherical curvature curved in two dimensions to reduceoff-axis aberrations.

Thus, using curved lens arrays may improve industrial design of HMDs.For example, using curved lens arrays opens many new opportunities todesign a slim device with a face-hugging form factor that may beimpossible with current HMD's. As used herein, a lenslet refers to asmall lens that is part of a lens array, also referred to as a lensletarray, including many other lenslets. In some examples, the curved lensarray may be a planar surface that has been flexed or thermo-formed intoa curved design. For example, the planar surface may include a planararray of lenses, with each of the lenses having a one or more differentshapes. Thus, the planar array of lenses may be initially manufacturedas a planar surface and then formed using any suitable technique into acurved design for ease of manufacture.

In some examples, the distance between the lenses in the curved lensarray and the display can be adjusted for near-sighted users to enablethem to see the display without glasses. In some examples, adjusting fornear-sightedness may include adjusting the lens-to-display distancewhile maintaining concentricity and changing the curvature of thedisplay to match the adjusted distance. For example, the curved screenand the curved lens array can be mechanically paired for changing thelens array-to-display spacing while preserving concentricity. In someexamples, a mechanically flexible organic light emitting diode (OLED)display may be used for the curved screens 106A and 106B with a fixedcurved lens array. In some examples, both the curved lens array and thecurved screen can be mechanically flexible. For example, having both thecurved lens array and the curved screen flexible may allow reshaping ofthe design while preserving lens-to-display concentricity. In someexamples, one or more lenslets of the curved lens array can beelectrically focus-tunable or dynamically switchable. For example,focus-tunable lenses can change focal length within the certain rangecontinuously depending on an applied voltage or current pattern.Switchable lenses can have several discrete focal lengths that can beselected electronically. In some examples, the focus-tunable lenses orthe dynamically switchable lenses can be liquid crystal based, membranebased, or electro-wetting based. The use of electrically focus-tunableor dynamically switchable lenslets may enable either multi-plane orvolumetric virtual rendering. Multi-plane or volumetric virtualrendering may be used for eliminating a convergence-accommodationconflict.

In some examples, the curved lens array may be replaceable. For example,the curved lens array may include a manually-operated design thatenables the user to replace the lens array with another lens array. Areplaceable lens array may allow for prescription lenses, such thatusers with visual impairment can look directly through a customizedcorrected VR without eyeglasses while maintaining the thin-format of thepresent design. Thus, users that normally wear prescription glasses maynot need to wear any glasses with curved lens arrays for improvedeyeglass-free visual comfort.

The diagram of FIG. 1 is not intended to indicate that the examplehead-mounted display 100 is to include all of the components shown inFIG. 1. Rather, the example head-mounted display 100 can be implementedusing fewer or additional components not illustrated in FIG. 1 (e.g.,additional layers, filters, lens elements, dimensions, etc.).

FIG. 2 is a diagram illustrating an example curved display for one eye.The example curved display is referred to generally by the referencenumber 200 and can be implemented in the computing device 1600 below inFIG. 16 using the method 1500 of FIG. 15 below.

The example of FIG. 2 includes similarly numbered elements from FIG. 1.For example, FIG. 2 shows a curved screen 106 and lens array 104 curvedalong the cylinders concentric with an eye center 202 of the viewer'seye 102. For example, the eye center 202 may correspond to the fovea ofthe eye. The fovea, or fovea centralis, is a small, central pit composedof closely packed cones in the center of the retina of the eye and isresponsible for sharp central vision. The curved display 200 includes alens array 104 with a number of lenses 204, also referred to herein aslenslets, and a curved screen 106. The curved screen 106 is divided intoa number of elemental images 206. For example, each elemental image 206may be rectangular region associated with a particular lens 204 in thelens array 104. Each elemental image 206 may be centered horizontallywith the correspondent lens 204.

A pupillary axis 110 is shown extending from the eye center 202 throughthe pupil of the eye 102 and towards the middle of the curved screen106.

As shown in FIG. 2, the curved screen 106 may be logically divided intorectangular regions, referred herein as elemental images 206, to beviewed by the eye 102 through a number of lenses 204. Each elementalimage 206 may be magnified by a single lens 204 of the curved lens array104. For example, each elemental image 206 may be magnified by a lenssuch that a viewer looking through that lens may see a magnified versionof the image appearing to be at the virtual distance from the lens.Together, the elemental images 206 may create a rectangular grid on thecurved screen 106. For example, the elemental images 206 may have beenrendered using any of the methods 1300 or 1500 described below. As shownusing the pupillary axis, light emitted by each of the elemental images206 of the curved screen 106 pass through one of the lenses 204 beforereaching the pupil of the eye 102. In some examples, the lenses 204 maybe heterogeneous. For example, the lenses 204 may have a different size,shape, curvature, etc. In some examples, the lenses 204 may be freeform, or having a surface that is not spherical in shape. For example,the shape of the lenses 204 can be electromechanically adjustable. Asalso shown in FIG. 2, when the screen is curved, the associatedelemental images 206 may also be rendered for the curved surface.

The diagram of FIG. 2 is not intended to indicate that the examplecurved virtual reality display 200 is to include all of the componentsshown in FIG. 2. Rather, the example curved virtual reality display 200can be implemented using fewer or additional components not illustratedin FIG. 2 (e.g., additional lenses, screens, elemental images, filters,dimensions, etc.).

In addition, although FIGS. 1 and 2, in addition to Figs. below, showcurving very close to cylindrical surfaces concentric to the eyerotation center 202, in some examples, other curved surfaces may be usedwith similar design principles. First, the use of other curved surfacesmay include selection of ideal eye position(s). Second, the use of othercurved surfaces may include allocating elemental images 206 to increaseuse of the surface of the screen 106. Third, the use of other curvedsurfaces may include selecting design parameters for best desiredresolution and tolerable viewing zone size.

FIG. 3A is a diagram illustrating an example viewing zone for a singlelens. The example viewing zone is referred to generally by the referencenumber 300A and can be implemented in the computing device 1600 below inFIG. 16 using the method 1500 of FIG. 15 below.

The example viewing zone 300A is shown with respect to one lens 204 of alens array. The viewing zone 300A is associated with the lens 204 and anassociated elemental image 206. The viewing zone 300A is also associatedwith a virtual image 304 that can be seen through lens 204.

As shown in FIG. 3A, the virtual image 304 may be seen within viewingzone 300A when viewing elemental image 206 through the lens 204. Thelens 204 in the array may act as a simple magnifier producing thevirtual image 304 of its corresponding elemental image 206 at distancedefined by the spacing and lens properties. In some examples, the lensproperties can be adjustable, such as with free form lenses. The viewingzone 300A of a single lens 204 may thus be a region in space where theviewer can see only the correspondent elemental image pixels through theaperture of the corresponding lens 204. For example, the viewing zone300A may be an area where pixels from other elemental images are notvisible through the given lens 204. In some examples, the shape of theviewing zone 300A may be defined by lens properties, elemental imagesize and mutual position of the lens and elemental image. For example,lens properties may include focal length, aperture, etc.

In some examples, a minimum viewing zone size for a thin HMD can becalculated based on an eye radius and eye pupil entrance. In someexamples, the viewing zone may have a box width based on a distance foran eye rotation center. For example, for static designs, the eye reliefand shape of the viewing zone may allow for an approximately 13millimeter eye box width at 6 millimeters from the eye rotation center.The 6 mm may be calculated based on a 13 millimeter eye radius minus aseven millimeter eye pupil entrance. In some examples, an HMD mayinclude an eye position tracker to track the position of the eyes of auser and a viewing zone comprising a box width at each pupil based on adistance from an eye rotation center or an error of margin of the eyeposition tracker. For example, the box width at the pupil may be largeenough to contain the pupil plus an error margin of the tracker. In someexamples, the box width can be parameterized as a box width at thecenter of rotation. In some examples, an apparatus can be designed for aviewing zone with the pupil size of the user taken into account plus theerror margin. For example, the pupil size may be five or sixmillimeters. The error margin may be based solely on the tracker. Insome examples, if eye position tracking is available, then the viewingzone size at the intended eye relief can be reduced to either the sizeof an average adult pupil or the error margin of the eye pupil tracker,whichever is greater. For example, the average adult pupil size may beapproximately five to six millimeters. Thus, an HMD with eye trackingmay use a viewing zone of at least a greater of 5 millimeters box widthat 6 millimeters from an eye rotation center or an error of margin ofthe eye position tracker.

The diagram of FIG. 3A is not intended to indicate that the exampleviewing zone 300A is to include all of the components shown in FIG. 3A.Rather, the example viewing zone 300A can be implemented using fewer oradditional components not illustrated in FIG. 3A (e.g., additionallenses, elemental images, virtual images, etc.).

FIG. 3B is a diagram illustrating an example compound viewing zone forthree lenses. The example compound viewing zone is referred to generallyby the reference number 300B and can be implemented in the computingdevice 1600 below in FIG. 16 using the method 1500 of FIG. 15 below.

The example compound viewing zone 300B is shown with respect to one lens204 of a lens array. The compound viewing zone 300B includes a number ofoverlapping viewing zones 300A, with associated lenses 204.

As shown in FIG. 3B, the compound viewing zone 300B may be an area ofintersection of three viewing zones 300A. In some examples, a resolutionof a display can be defined as the total number of pixels visiblethrough all lenses 204 across the display. Thus, the number of elementalimage pixels visible through any single lens 204 can be used to estimatethe resolution. The number of elemental image pixels visible may alsodepend on how close the viewer's eye is assuming the eye remains withinthe viewing zone. For example, the closer the eye is to the lens, thelarger the field of view (FOV) through that lens, the more pixels theeye can see. Thus, there is a direct tradeoff between eye relief,viewing zone size, and the number of pixels visible to the observer. Insome examples, these three factors can be balanced to achieve animproved design. For example, an eye relief limit may be based on ashape of a viewing zone of the HMD.

The FOV may be the angle through which the viewer is able to see thedisplay. In some examples, increases in FOV do not increase the devicethickness. Rather, additional lenses, and display surface, may be addedto the perimeter of the design without increasing device thickness.

The diagram of FIG. 3B is not intended to indicate that the examplecompound viewing zone 300B is to include all of the components shown inFIG. 3B. Rather, the example compound viewing zone 300B can beimplemented using fewer or additional components not illustrated in FIG.3B (e.g., additional viewing zones, lenses, etc.).

FIG. 4 is a diagram illustrating an example compound viewing zone for alarger number of lenses. The example compound viewing zone is referredto generally by the reference number 402 and can be implemented in thecomputing device 1600 below in FIG. 16 using the method 1500 of FIG. 15below.

FIG. 4 shows the example compound viewing zone 402 with respect to 18lenses 204 of a lens array 104. The example of FIG. 4 also includes anassociated curved screen 206. A boundary line 404 of the compoundviewing zone 402 indicates a boundary beyond which an eye is not allowedto move. A minimal eye relief 406 is indicated by an arrow.

As shown in FIG. 4, for a larger number of lenses 204 the distancebetween viewing zone 400 and lens array 104 may be bounded by theboundary 404 of the compound viewing zone 402 that is closest to the eyeand gives a good estimate for minimum eye relief 406. For example, for aconcentric cylindrical design, the minimum allowable eye relief 406 maybe defined by a distance between the center of rotation of the eye andthe boundaries of the viewing zones making up the compound viewing zone402.

Changing the viewing zone implies changing the eye relief and the eyebox shape. In some examples, the shape of the compound viewing zone 402may be taken into account when optimizing for the system parameters,including eye relief and eye box shape, among other parameters. Forexample, changing the lens aperture, the focal length, or thelens-screen spacing may affect the shape of viewing zone and number ofpixels visible through a given lens. In some examples, when number oflenses is small, the exact shape of the viewing zone can be used toreduce minimum possible eye relief and thus increase the perceivedresolution of the system by allowing smaller eye relief. Eye relief isdistance from the last surface of an eyepiece within which a user's eyecan obtain a full viewing angle.

In some examples, a minimum allowable eye relief 406 can be calculatedbased on the shape of the viewing zone 402. For example, with designshaving fewer lenses and thus a smaller combined FOV, the intersection ofthe viewing zones may form a different shape than the approximatecylinder shown above. In some examples, if an exact shape of the viewingzone is known, then the minimum possible eye relief can be reduced toincrease the perceived resolution of the system. In some examples, ahuman retina-matched design may be used. For example, ordinarily, if aneye is too close to the display, a ‘broken image’ may be observed. In afinely-tuned human retina-matched design, these image formation failurescan be hidden in the peripheral vision of the human retina. Since humanvision has very poor resolution beyond a small angle, a user may notnotice the image formation failures in peripheral vision.

The diagram of FIG. 4 is not intended to indicate that the examplecompound viewing zone 400 is to include all of the components shown inFIG. 4. Rather, the example compound viewing zone 400 can be implementedusing fewer or additional components not illustrated in FIG. 4 (e.g.,additional lenses, viewing zones, etc.).

FIG. 5 is a diagram illustrating an example thick lens model. Theexample thick lens model is referred to generally by the referencenumber 500 and can be implemented in the computing device 1600 below inFIG. 16 using the method 1500 of FIG. 15 below.

The example thick lens model 500 is shown with respect to one lens 204of a lens array. A thick lens, as used herein, refers to a lens with athickness measured by a distance along the optical axis between the twosurfaces of the lens that is not negligible compared to the radii ofcurvature of the lens surfaces. The thick lens model 500 includes an eyecenter 202 of an eye 102 looking through a lens 204 at a curved screen106. A distance 502 between the eye 102 and lens 204 is indicated by anarrow. A distance 504 between the lens 204 and a chord 506 intersectingthe curved screen 106 is indicated by another arrow. A length 508 of thechord 506 is indicated by an arrow. A width 510 of the lens 204 is alsoindicated by an arrow. A screen surface radius 512 indicates thedistance from the eye center 202 to the curved screen 106.

As shown in FIG. 5, the thick lens model 500 may be used for computingcorrections for screen curving. For example, spacing values andelemental image size may be corrected according to thick lens equationsand measured with respect to two principal planes. In some examples, thelength 508 of the chord 506 can be equal to the size of an elementalimage associated with the lens 204 to correct for screen curving. Insome examples, a parameter optimization may be performed on lens pitch,effective resolution, FOV, lens-to-display spacing, ‘total thickness,’and other metrics. For example, total thickness may include a displaythickness, a distance between lens array and display, and a lens arraythickness. In some examples, the parameters can be selected to minimizethickness and maximize resolution for a given display. In some examples,since retinal blur cues are not needed, larger lenses may be used thantypically used to detect retinal blur cues. In some examples, the lensarray pitch, focal length, and display-to-lens spacing may be selectedbased on any of the considerations discussed above.

The diagram of FIG. 5 is not intended to indicate that the example thicklens model 500 is to include all of the components shown in FIG. 5.Rather, the example thick lens model 500 can be implemented using feweror additional components not illustrated in FIG. 5 (e.g., additionallenses, dimensions, screens, etc.).

FIG. 6A is a diagram illustrating an example lens design using asphericsurfaces. The example lens design is referred to generally by thereference number 600A and can be implemented in the computing device1600 below in FIG. 16 using the method 1500 of FIG. 15 below.

The example lens design 600A is shown with respect to one lens 302 of alens array. The lens design 600A is shown receiving a directed form oflight 602. The lens design 600A includes an aspheric lens 604. FIG. 6Aalso shows an iris plane 606 and lens plane 608 of a model eye 610. Themodel eye 610 also includes an eye center 202.

As described in greater detail above, the basic physical designparameters such as lenslet pitch, virtual object distance,screen-lenslet spacing, and eye relief may be first determined tosatisfy the desired eye-box, magnification, elemental image size,spatial resolution, and field of view (FOV). Afterwards, the individuallens parameters can be set to be variables while an optimizer caniterate to find the best combination of parameters that minimizes apreset objective function. For example, the individual lens parametersmay include radii of curvatures, position, thickness, conic, high-ordersurface terms, etc.

As shown in FIG. 6A, in some examples, the lens design 600A may useaspheric lenses 604 to eliminate spherical aberrations. For example, thespherical aberrations may be caused by the curvature of the screen. Insome examples, the individual lens parameters of each of the asphericlenses 604 can be adjusted to remove spherical aberrations.

In some examples, to improve the accuracy of the optical design, arealistic eye model that reflects the human eye's sensitivity in thevisible spectrum can be used. For example, higher weight may be given togreen wavelengths. For example, wavelengths of approximate 0.555micrometers may be given a weight of 1.0, while wavelengths ofapproximately 0.47, 0.51, 0.61, and 0.65 may be given weights of 0.091,0.503, 0.503, and 0.107, respectively. In some examples, the eye modelmay also include varying visual acuity on the retina. As one example, avisual acuity of 0.3 milliradians or approximately 2.5 micrometers maybe given to the fovea. Thus, higher weight may be given to the fovearegion of the retina. In some examples, given rotation of the eye, alogarithmic function may be used to provide a weight of 1.0 to a 0degree vertical angle offset, a 0.2 weight to a 10 degree angle offset,and a 0.02 weight to a 45 degree angle offset. The human eye has fourkey dynamic parameters that can be considered in the eye model:position, rotation, iris aperture, and lens focus. For example, the eyemodel may have a focal length of approximately 17 millimeters in air. Insome examples, the eye model may have a designed focus at 1000millimeters with a focus range of 170 millimeters. A horizontal FOV forboth eyes may be 200 degrees, including 100 degrees outward and 60degrees inwards. A vertical FOV for the eyes may be 135 degrees, with 60degrees upward FOV and 75 degrees downward FOV. In addition, the eyemodel may include a design aperture of 3.6 millimeters with a range ofapproximately 2 to 8 millimeters in diameter.

The diagram of FIG. 6A is not intended to indicate that the example lensdesign 600A is to include all of the components shown in FIG. 6A.Rather, the example lens design 600A can be implemented using fewer oradditional components not illustrated in FIG. 6A (e.g., additionaldimensions, lenses, eye models, etc.).

FIG. 6B is a diagram illustrating an example lens design using free-formsurfaces. The example lens design is referred to generally by thereference number 600B and can be implemented in the computing device1600 below in FIG. 16 using the method 1500 of FIG. 15 below.

The example lens design 600B includes similarly numbered elements fromFIG. 6A. The lens design 600B includes light rays 602 from the samepoint source as FIG. 6A, but shown being focused using a free form lens612 instead.

As described in FIG. 6A above, aspheric lens surfaces can be used toeliminate spherical aberrations. However, the aspheric lens surfaces maybe rotationally symmetric. Thus, the aspheric lens surfaces may not beable to be optimized or curved for a given eye position. This may beespecially true for off-axis lenslets. In some examples, free-formoptics may be better suited for any given eye position as they arenon-symmetric and the optimizer can fully control the shape and curve oftheir surfaces by optimizing the coefficients of high-order polynomialsin both X and Y directions or as radial coefficients. Each lens in thelenslet array can thus be optimized separately so the elemental imagesare formed sharply on the retina.

In some examples, a multi-state focus tunable heterogeneous lensoptimization may include dynamic or switchable lens arrays to vary thevirtual image distance. A focus-tunable design may enable largerrendered depth of field and eliminate the vergence-accommodationconflict problem resulting in more natural scenery. In particular, thevergence-accommodation conflict forces a viewer's brain to unnaturallyadapt to conflicting cues and increases fusion time of binocularimagery, while decreasing fusion accuracy. More specifically, retinalblur is the actual visual cue driving the oculomotor response ofaccommodation, or adjustment of the eye's lens to focus on the desireddepth to minimize the blur. Retinal disparity is the visual cue thatdrives vergence. In addition, there is a dual and parallel feedback loopbetween vergence and accommodation in which one becomes a secondary cueinfluencing the other. In typical HMD designs, the virtual image may befocused at a fixed depth away from the eyes, while the depth of virtualobjects and thus binocular disparity may vary with the content, whichmay result in conflicting information within vergence-accommodationfeedback loops. The vergence-accomodation conflict can cause visualfatigue in users, particularly after prolonged use. A focus-tunabledesign allows for presenting multiple image planes at different virtualdistances creating an appearance of the volumetric image rather thanjust a single image surface. In some examples, a programmable lens arraycan also be reconfigured to compensate for user's own opticalaberrations eliminating the need for visual-aid glasses when using thinVR HMD. For example, the curving and shape of individual lenses may beadjusted based on the user's known optical aberrations.

The diagram of FIG. 6B is not intended to indicate that the example lensdesign 600B is to include all of the components shown in FIG. 6B.Rather, the example lens design 600B can be implemented using fewer oradditional components not illustrated in FIG. 6B (e.g., additionaldimensions, lenses, eye models, etc.).

FIG. 7A is a diagram illustrating an example heterogeneous free-formlenslet array. The example heterogeneous free-form lenslet array isreferred to generally by the reference number 700A and can beimplemented in the computing device 1600 below in FIG. 16 using themethod 1500 of FIG. 15 below.

The example heterogeneous free-form lenslet array 700A is shown with sixlenslets 702. The heterogeneous free-form lenslet array 700A is shownmagnifying individual rays of light.

As shown in FIG. 7A, the heterogeneous free-form lenslet array 700A mayinclude differently shaped heterogeneous lenslets to reduce aberrations.In some examples, lenslet arrays may include periodic patterns where thelenslets are identical across the array regardless of how the eye ispositioned or oriented. For example, the principal planes of lenslets ofthe patterned design can be replicated along an arc of a curvatureradius based on an eyeball radius, an eye relief, and a lens thickness.The curvature radius R may be calculated using the equation:R=Eyeball Radius+Eye Relief+HalfLensThickness  Eq. 1However, in some examples, using identical lenslets may result insignificant aberrations due to the optical path differences in theoff-axial lenslets as opposed to the on-axis one aligned with the eye'saxis. Therefore, a heterogeneous design in which each lenslet isoptimized separately can be used to reduce or balance all aberrationsand deliver sharper and higher-resolution scenery to users, as describedin FIG. 7B below.

The diagram of FIG. 7A is not intended to indicate that the exampleheterogeneous free-form lenslet array 700A is to include all of thecomponents shown in FIG. 7A. Rather, the example heterogeneous free-formlenslet array 700A can be implemented using fewer or additionalcomponents not illustrated in FIG. 7A (e.g., additional lenses,dimensions, light rays, eyes, etc.).

FIG. 7B is a diagram illustrating an example heterogeneous free-formlenslet array adapted for eye rotation. The example heterogeneousfree-form lenslet array is referred to generally by the reference number700B and can be implemented in the computing device 1600 below in FIG.16 using the method 1500 of FIG. 15 below.

The example heterogeneous free-form lenslet array 700B includes a sixlenses 702. The heterogeneous free-form lenslet array 700B may have beenoptimized for eye rotation.

As shown in FIG. 7B, the heterogeneous free-form lenslet array 700B isshown magnifying light rays towards a rotating eye 704. Since the humaneye is realistically continuously scanning scenery, the optical designcan be either optimized for a fixed eye rotation or optimized such thatthe best rendered resolution is always aligned with the fovea region.For example, in a fixed eye rotation, the heterogeneous free-formlenslet array 700B may be optimized such that the rendered object isoptimal for the whole screen. In some examples, the best renderedresolution may be always aligned with the fovea region or eye centersince the peripheral regions may be of low resolution. For example, thelenslets may be adapted in either fixed adaptation at the designingstage or in a real-time adaptation using dynamic optics to account forvarious eye rotations. In some examples, the optimizer may elect to bendthe off-axial lenses as shown in FIG. 7B to improve performance.

The diagram of FIG. 7B is not intended to indicate that the exampleheterogeneous free-form lenslet array 700B is to include all of thecomponents shown in FIG. 7B. Rather, the example heterogeneous free-formlenslet array 700B can be implemented using fewer or additionalcomponents not illustrated in FIG. 7B (e.g., additional lenses,dimensions, light rays, eyes, etc.).

FIG. 8 is a diagram illustrating an example system for rendering 3Dcontent to a virtual plane. The example system is referred to generallyby the reference number 800 and can be implemented in the computingdevice 1600 below in FIG. 16 using the method 1500 of FIG. 15 below.

The example system 800 is shown with respect to one eye center 202. Thesystem 900 includes a curved lens array 104 magnifying a set ofelemental images from a curved screen 106. The system 800 furtherincludes a virtual plane 802 and three dimensional content 804.

In the example of FIG. 8, the elemental images corresponding to threedimensional content 804 presented by the curved screen 106 may have beenrendered to a single virtual plane 802. For example, the virtual plane802 may be emulating a virtual plane in a stereo head mounted display.

The diagram of FIG. 8 is not intended to indicate that the examplesystem 800 is to include all of the components shown in FIG. 8. Rather,the example system 800 can be implemented using fewer or additionalcomponents not illustrated in FIG. 8 (e.g., additional eyes, curved lensarrays, curved screens, virtual surfaces, content, etc.).

FIG. 9 is a diagram illustrating an example system for rendering 3Dcontent to a cylindrical rendering surface. The example system isreferred to generally by the reference number 900 and can be implementedin the computing device 1600 below in FIG. 16 using the method 1500 ofFIG. 15 below.

The example system 900 is shown with respect to one eye center 202. Thesystem 900 includes a curved lens array 104 magnifying a set ofelemental images from a curved screen 106. The system 900 includes acylindrical virtual surface 902 and three dimensional content 804.

As shown in FIG. 9, since the FOV of the HMD containing the curvedscreen 106 and curved lens array 104 may be large, a betterapproximation of the pixel colors can be achieved by selecting a morecomplex but still simple set of virtual surfaces instead of a singleplane 802 as used in FIG. 8 above. For example, the cylindrical virtualsurface 902 may be a set of flat planes angled to form an approximatelycylindrical surface. The virtual cylindrical surface 902 may be used torender three dimensional content 804 onto the curved screen 106 to thusprovide a more accurate presentation.

The diagram of FIG. 9 is not intended to indicate that the examplesystem 900 is to include all of the components shown in FIG. 9. Rather,the example system 900 can be implemented using fewer or additionalcomponents not illustrated in FIG. 9 (e.g., additional eyes, curved lensarrays, curved screens, virtual surfaces, content, etc.).

FIG. 10 is a diagram illustrating an example mapping of an image planeonto an elemental image. The example mapping is referred to generally bythe reference number 1000 and can be implemented in the computing device1600 below in FIG. 16 using the method 1500 of FIG. 15 below.

The example mapping 1000 is shown with respect to one eye center 202 ofan eye. The mapping 1000 includes a center of projection 1002. Themapping 1000 also includes a center of lens 1004 shown with respect to alens 204 in a curved lens array 104. The mapping 1000 also includes anelemental image pixel 1006 presented on a curved screen 1006. Themapping 1000 further includes a virtual image 1008 including a virtualimage of the pixel 1010. The mapping 1000 further includes a virtualplane 1012 and a pixel 1014 on the virtual plane 1012 corresponding toelemental image pixel 1006 and virtual image pixel 1010.

As shown in FIG. 10, the mapping 1000 may be used to map a virtual plane1012 to an elemental image presented on the curved screen 106. In someexamples, the mapping 1000 may be used for a two ray casting operationwhere rays intersect with simple surfaces. For example, for eachelemental image, a processor can select a center of projection 1002. Thecenter of projection 1002 may be determined from a range of possiblelocations as described with respect to FIG. 11 below. For each pixel1006 of elemental image, the processor can find the projection of pixel1006 through the lens center 1004 onto virtual image 1008. The resultmay be virtual image pixel 1010, or the intersection of ray of lightpassing through lens center 1004 with virtual image 1008. The processorcan then find the projection of virtual image pixel 1010 through theprojection center 1002 onto the virtual image plane 1012. The result maybe point 1014. The processor may then fetch the color from point 1014and set it to elemental image pixel 1006. Each virtual image 1008 ofelemental image 1006 creates a perspective projection view centered atcenter of projection 1002. Views from multiple elemental images can beused to create multiple views of a virtual plane 1014 in the vicinity ofa viewer's eye. In some examples, the positions of the center ofprojection 1002 may be pre-selected and fixed when eye tracking is notavailable.

The diagram of FIG. 10 is not intended to indicate that the examplemapping 1000 is to include all of the components shown in FIG. 10.Rather, the example mapping 1000 can be implemented using fewer oradditional components not illustrated in FIG. 10 (e.g., additionaldimensions, pixels, virtual surfaces, etc.).

FIG. 11 is a diagram illustrating an example range of locations for acenter of projection when rendering elemental images. The example rangeof locations is referred to generally by the reference number 1100 andcan be implemented in the computing device 1600 below in FIG. 16 usingthe method 1500 of FIG. 15 below.

The example range of locations 1100 is shown with respect to one eyecenter 202 of an eye. The range of locations 1100 includes a center ofprojection 1002 located along a range 1102 of locations of the center ofprojection 1002. The range 1102 extends from the eye center 202 to thelens center 1004 of one of the lenses 204 of a curved lens array 104.FIG. 11 also includes a curved screen 106 and virtual image 1008associated with virtual plane 1012.

As shown in FIG. 11, the range of locations 1102 for the center ofprojection 1002 extends from the eye center 202 to the center 1004 ofone of a lens 204. In some examples, as discussed above, when eyetracking is not available, the center of projection 1002 canpre-selected and fixed. Thus, where exact viewer eye position isassumed, the center of projection 1002 can be located on the line 1102connecting the center of the lens 1004 and elemental image at a fixeddistance from the lens, or more specifically, the lens principal plane.In some examples, the fixed distance can be equal or close to the idealeye relief. For example, the ideal eye relief may defined by theintended eye location for which perspective projections of virtualimages will give absolutely correct fused image. The actual value of thefixed distance may be an adjustment parameter or calibration parameterof the system allowing to compensate for deviations in spacing and lensfocal length. Thus, the center of the lens 1004 can be selected from arange 1102 of locations allowing tradeoff between amount of parallax andprojection distortions or plane swim. For example, if an image isrendered for a specific eye position, then the image plane may appeardistorted when your eye moves form that position, which is referred toas plane swim. In some examples, multiple image planes may all shift ordistort when and eye of a user moves from the intended eye position.Therefore, either the pixels for the eye position may be updated or theprojection center may be moved towards the optical center of the lensaccordingly.

In some examples, when eye tracking is available, the center ofprojection 1002 can be selected in the vicinity of the center of the eyeaperture. Furthermore, in some examples, it may be possible to identifylenses and regions of virtual surfaces that deliver image to the foveaof the eye and use multi-resolution shading or foveated renderingtechniques. For example, less resolution may be provided to areas of animage that are less visible due to less light or areas that are in theperiphery of human vision, respectively.

The diagram of FIG. 11 is not intended to indicate that the examplerange of locations 1100 is to include all of the components shown inFIG. 11. Rather, the example range of locations 1100 can be implementedusing fewer or additional components not illustrated in FIG. 11 (e.g.,additional centers of projection, dimensions, pixels, virtual surfaces,etc.).

FIG. 12 is a block diagram illustrating an example method of renderingelemental images using ray tracing. The example method is generallyreferred to by the reference number 1200 and can be implemented in thecomputing device 1500 below in FIG. 15. For example, the ray tracingmethod 1200 can be used in the method 1400 and the elemental imagegenerator of the computing device 1500 of FIG. 15 below.

In some examples, the location of elemental images projected onto aretina may vary with a rotated eye. Therefore, an accurate ray tracingmethod 1200 can be used to deliver sharp VR rendering for heterogeneouslens array.

At block, 1202, a processor can receive a virtual image to be used torender the elemental images. The virtual image may include threedimensional content to be displayed to a user in a head mounted display.For example, the virtual image may be a single plane of a stereo paircreated by rendering 3D models.

At block 1204, the processor can establish a correspondence between thevirtual image and the retina. For example, the processor can determine acorrespondence between the virtual image and the retina for a set ofgiven eye parameters without including any optics. In some examples, thecorrespondence may be a mapping between the virtual image and retina.The mapping may be used to determine content on the retina for a givenset of eye parameters. In some examples, the mapping may be stored in alook-up table containing mappings for a variety of eye parametercombinations. Thus, given an optical design and an eye model, theprocessor can accurately trace rays for various eye parameters andgenerate in advance look-up tables storing the correspondence mappinginformation between the screen and the retina.

At block 1206, the processor back-projects the retinal pixels through aheterogeneous free-form lens array to the screen using the precomputedlook-up table so that the elemental images content on the screen iscomputed. In some examples, for each traced ray, the look-up table canprovide the launching screen pixel (x_(o), y_(o)), the direction of theray (P_(x), P_(y)), the landing location on retina (x_(i), y_(i),z_(i)), the ray's wavelength (λ), the ray's intensity (I) in case ofabsorbing/attenuating martials, and whether or not that traced ray isvignetted.

As shown by arrow 1208, the method 1200 can be repeated in real-time toaccurately reflect any change of eye parameters or the scenery on therendered VR content. In some examples, a change of eye parameters can beestimated using an eye pupil tracker. For example, the eye parameterscan be accurately estimated in real-time using eye pupil tracker andutilized to determine which look-up table to consider for synthesizingthe elemental images that best render the fused sharp VR image.

This process flow diagram is not intended to indicate that the blocks ofthe example method 1200 are to be executed in any particular order, orthat all of the blocks are to be included in every case. Further, anynumber of additional blocks not shown may be included within the examplemethod 1200, depending on the details of the specific implementation.

FIG. 13 is a table comparing example spot diagrams of field pointsaligned with a fovea of a rotated eye for identical and heterogeneouslens designs. The table is generally referred to by the reference number1300. For example, the spot diagrams 1300 may indicate resolutionperformance using the head mounted display 100 of FIG. 1, the method1400 of FIG. 14, the head mounted display 1526 of FIG. 15 below, with anidentical array of aspheric lenses 604 of FIG. 6A versus an array offree form lenses 612 of FIG. 6B.

FIG. 13 includes a row of various eye rotation values 1302, and acorresponding sets of identical lens spot diagrams 1304 andheterogeneous lens spot diagrams 1306, referenced using the referencenumbers 1308-1318 and 1320-1330, respectively. FIG. 13 illustrates anexample of the resolution improvement made by using a heterogeneous lensarray design as opposed to the resolution delivered by identical lensarray design. The spot diagrams 1308-1330 represent the point spreadfunction (PSF) for field points aligned with the fovea of an eye forvarious eye rotations 1302. The RMS radius values measured inmicrometers (μm) quantify the delivered resolution.

As shown in FIG. 13, the RMS radius values for identical lenses atvarious eye rotations 1308-1318 range from 12.510 at zero degreesrotation to 1006.74 at 32.45 degrees rotation. By contrast, the RMSradius values for heterogeneous lenses at various eye rotations1320-1330 range from 12.445 at zero degrees to a much smaller 23.260 at32.45 degrees. Thus, although identical 1304 and heterogeneous 1306 lensarrays may perform similarly at zero degrees of rotation, theheterogeneous 1306 lens array outperforms the identical 1304 lens arrayat every angle of rotation, with increasingly improved performance asthe rotation angle increases.

FIG. 14 is a flow chart illustrating a method for generating elementalimages to be presented via a head mounted display. The example method isgenerally referred to by the reference number 1400 and can beimplemented in the processor 1502 of the computing device 1500 of FIG.15 below, or the computer readable media 1600 of FIG. 16 below. In someexamples, the method 1400 may be used to generate elemental images to bedisplayed via the head mounted display 100 of FIG. 1 above or the headmounted display 1526 of FIG. 15 below.

At block 1402, a processor receives an image to be presented and avirtual distance from eyes of a viewer. For example, the image may be athree dimensional scene to be displayed. In some examples, the image maybe a three dimensional model. The virtual distance may be a distancebetween a virtual surface and an eye center. For example, the virtualdistance may be the screen surface radius 512 described with respect toFIG. 5 above.

At block 1404, the processor renders a stereo view of the image for eachof the eyes at a virtual surface located at the virtual distance. Insome examples, the virtual surface may be a plane. For example, thevirtual surface may be similar to the plane 802 describe in FIG. 8above. The virtual surface may be a plane of an emulation of atraditional stereo HMD with two flat screen planes located at a givenvirtual distance from the viewer's eyes. In some examples, the virtualsurface may be a cylindrical surface or a piecewise linear approximationof a cylindrical surface. For example, the virtual surface may be thecylindrical surface described with respect to FIG. 9 above.

At block 1406, the processor maps pixels for each of the stereo viewsfrom the virtual surface to elemental images of a per-eye display usinga per-lens projection model. In some examples, the per-lens projectionmodel may include a perspective projection of virtual image with acenter of projection assigned to a particular individual lens. Forexample, the per-lens projection model may include pixel 1014 on thevirtual plane 1012 being projected to a pixel 1006 on the screen 106using optical center 1004 of the lens 204 as the center ofaforementioned projection, as shown in FIG. 10 above. In some examples,the pixels may be mapped using a pixel shader.

At block 1408, the processor pre-warps the elemental images based on aper-lens distortion model to compensate for a lens distortion.Distortion as used herein refers to a deviation from rectilinearprojection, which is a projection in which straight lines in a sceneremain straight in an image. For example, the per-lens distortion modelmay include a polynomial approximation of deviation of pixel coordinatesfrom a rectilinear projection.

At block 1410, the processor sends the pre-warped elemental images to ahead mounted display to be displayed. For example, the head mounteddisplay may be the head mounted display of FIG. 1 above or FIG. 16below.

This process flow diagram is not intended to indicate that the blocks ofthe example process 1400 are to be executed in any particular order, orthat all of the blocks are to be included in every case. For example,blocks 1404-1408 can be implemented in any order or granularity that isnot breaking data dependency. In some examples, blocks 1406 and 1408 canbe combined by using a warping function to re-map the rays and use thederivatives of the warping function for better sampling of a texture ofthe virtual surface. Further, any number of additional blocks not shownmay be included within the example process 1400, depending on thedetails of the specific implementation. For example, the method 1400 mayalso include receiving eye tracking data. For example, the processor mayreceive the eye tracking data from a pupil tracker. In some examples,the processor can render the stereo views and map the pixels usingmulti-resolution shading. For example, multi-resolution shading may beused to save processing resources by rendering areas that are mappedaway from the fovea in a lower resolution. In some examples, theprocessor can render the stereo views using foveated rendering. Forexample, the pixels may be rendered with higher resolution towards andinside the fovea.

In some examples, the processor can estimate an eye parameter inreal-time using an eye pupil tracker and retrieving a mapping from alook-up table based on the estimated eye parameter. For example, thepixels can be mapped using the ray tracing method 1200 described withrespect to FIG. 12 above. In some examples, the processor can use themapping to generate the elemental images. In addition, in some examples,the pixels may alternatively be mapped using a two ray castingoperation. For example, the processor can trace rays for a plurality ofeye parameters based on a design of the head mounted display and an eyemodel to generate a mapping between a screen of the head mounted displayand a retina of each of the eyes and storing the mapping in a look-uptable. In some examples, the processor can then estimate an eyeparameter in real-time using an eye pupil tracker and retrieve themapping from a look-up table based on the estimated eye parameter togenerate the elemental images.

Referring now to FIG. 15, a block diagram is shown illustrating anexample computing device that can render and present elemental imagesvia a head mounted display. The computing device 1500 may be, forexample, a laptop computer, desktop computer, tablet computer, mobiledevice, or wearable device, among others. In some examples, thecomputing device 1500 may be a smart camera or a digital securitysurveillance camera. The computing device 1500 may include a centralprocessing unit (CPU) 1502 that is configured to execute storedinstructions, as well as a memory device 1504 that stores instructionsthat are executable by the CPU 1502. The CPU 1502 may be coupled to thememory device 1504 by a bus 1506. Additionally, the CPU 1502 can be asingle core processor, a multi-core processor, a computing cluster, orany number of other configurations. Furthermore, the computing device1500 may include more than one CPU 1502. In some examples, the CPU 1502may be a system-on-chip (SoC) with a multi-core processor architecture.In some examples, the CPU 1502 can be a specialized digital signalprocessor (DSP) used for image processing. The memory device 1504 caninclude random access memory (RAM), read only memory (ROM), flashmemory, or any other suitable memory systems. For example, the memorydevice 1504 may include dynamic random access memory (DRAM).

The memory device 1504 can include random access memory (RAM), read onlymemory (ROM), flash memory, or any other suitable memory systems. Forexample, the memory device 1504 may include dynamic random access memory(DRAM). The memory device 1504 may include device drivers 1510 that areconfigured to execute the instructions for device discovery. The devicedrivers 1510 may be software, an application program, application code,or the like.

The computing device 1500 may also include a graphics processing unit(GPU) 1508. As shown, the CPU 1502 may be coupled through the bus 1506to the GPU 1508. The GPU 1508 may be configured to perform any number ofgraphics operations within the computing device 1500. For example, theGPU 1508 may be configured to render or manipulate graphics images,graphics frames, videos, or the like, to be displayed to a user of thecomputing device 1500.

The memory device 1504 can include random access memory (RAM), read onlymemory (ROM), flash memory, or any other suitable memory systems. Forexample, the memory device 1504 may include dynamic random access memory(DRAM). The memory device 1504 may include device drivers 1510 that areconfigured to execute the instructions for generating elemental images.The device drivers 1510 may be software, an application program,application code, or the like.

The CPU 1502 may also be connected through the bus 1506 to aninput/output (I/O) device interface 1512 configured to connect thecomputing device 1500 to one or more I/O devices 1514. The I/O devices1514 may include, for example, a keyboard and a pointing device, whereinthe pointing device may include a touchpad or a touchscreen, amongothers. The I/O devices 1514 may be built-in components of the computingdevice 1500, or may be devices that are externally connected to thecomputing device 1500. In some examples, the memory 1504 may becommunicatively coupled to I/O devices 1514 through direct memory access(DMA).

The CPU 1502 may also be linked through the bus 1506 to a displayinterface 1516 configured to connect the computing device 1500 to adisplay device 1518. The display device 1518 may include a displayscreen that is a built-in component of the computing device 1500. Thedisplay device 1518 may also include a computer monitor, television, orprojector, among others, that is internal to or externally connected tothe computing device 1500.

The computing device 1500 also includes a storage device 1520. Thestorage device 1520 is a physical memory such as a hard drive, anoptical drive, a thumbdrive, an array of drives, a solid-state drive, orany combinations thereof. The storage device 1520 may also includeremote storage drives.

The computing device 1500 may also include a network interfacecontroller (NIC) 1522. The NIC 1522 may be configured to connect thecomputing device 1500 through the bus 1506 to a network 1524. Thenetwork 1524 may be a wide area network (WAN), local area network (LAN),or the Internet, among others. In some examples, the device maycommunicate with other devices through a wireless technology. Forexample, the device may communicate with other devices via a wirelesslocal area network connection. In some examples, the device may connectand communicate with other devices via Bluetooth® or similar technology.

The computing device 1500 further includes a head mounted display 1526.For example, the head mounted display 1526 may include a curved screento display a plurality of elemental images. The head mounted display1526 may also include a curved lens array concentrically displaced infront of the curved screen to magnify the elemental images. For example,each of the plurality of elemental images is magnified by a differentlens in the curved lens array. In some examples, the curved lens arraymay have a lens array pitch and a display spacing based on a targetperceived resolution, a target field of view, a target total thickness,and a display pixel pitch. For example, the lens array pitch and adisplay spacing may be optimized for the target perceived resolution,target field of view, and target total thickness given an existingdisplay pixel pitch. In some examples, the curved lens array may be aheterogeneous array of freeform lenses. In some examples, the curvedlens array may include one or more flat sections, one or morecylindrically curved sections, or a combination thereof. In someexamples, the head mounted display may include a processor to displaythe elemental images. For example, the head mounted display 1526 may bea thin HMD with a curve display as described above in FIGS. 1-11 above.

The computing device 1500 further includes an elemental image generator1528. For example, the elemental image generator 1528 can be used togenerate elemental images to be presented on a head mounted display witha curved display. The elemental image generator 1528 can include areceiver 1530, a renderer 1532, a mapper 1534, a pre-warper 1536, and atransmitter 1538. In some examples, each of the components 1530-1538 ofthe elemental image generator 1528 may be a microcontroller, embeddedprocessor, or software module. The receiver 1530 can receive an image tobe presented and a virtual distance from eyes of a viewer. For example,the image may include a three dimensional scene to be displayed in thehead mounted display 1526. In some examples, the virtual distance may bea distance between a virtual surface and an eye center of the eyes of aviewer. The renderer 1532 can render a stereo view of the image for eachof the eyes at a virtual surface located at the virtual distance. Forexample, the virtual surface may be a virtual plane. In some examples,the virtual surface may be a cylindrical surface. In some examples, thevirtual surface may be a piecewise linear approximation of a cylindricalsurface. For example, the piecewise linear approximation of the virtualsurface may include a number of flat surfaces arranged in a cylindricalshape. The mapper 1534 can map pixels for each stereo view from thevirtual plane to elemental images of a per-eye display using a per-lensprojection model. For example, the per-lens projection model may includea perspective projection of a virtual image, as shown in FIG. 10 above.In some examples, the per-lens projection model may include a model ofelemental image distortion introduced by the particular lens. Forexample, the model of elemental image distortion may be a polynomialmodel. The use of an elemental image distortion model may yield a moreaccurate and practical calibration. In some examples, each elementalimages may be associated with a different lens of a curved lens array.In some examples, the mapper 1534 may be a pixel shader. The pre-warper1536 can pre-warp the elemental images based on a per-lens distortionmodel to compensate for a lens distortion. For example, each of thelenses in a curved lens array may be associated with a custom per-lensdistortion model to compensate for a lens distortion of each of thelenses of the curved lens array. In some examples, the lenses of thecurved lens array may be similar and a similar per-lens distortion modelmay be used. The transmitter 1538 can send the pre-warped elementalimages to a head mounted display to be displayed. For example, the headmounted display may be the head mounted display 1526.

The block diagram of FIG. 15 is not intended to indicate that thecomputing device 1500 is to include all of the components shown in FIG.15. Rather, the computing device 1500 can include fewer or additionalcomponents not illustrated in FIG. 15, such as additional buffers,additional processors, and the like. The computing device 1500 mayinclude any number of additional components not shown in FIG. 15,depending on the details of the specific implementation. Furthermore,any of the functionalities of the renderer 1532, the mapper 1534, thepre-warper 1536, and the transmitter 1538, may be partially, orentirely, implemented in hardware and/or in the processor 1502. Forexample, the functionality may be implemented with an applicationspecific integrated circuit, in logic implemented in the processor 1502,or in any other device. Furthermore, any of the functionalities of theCPU 1502 may be partially, or entirely, implemented in hardware and/orin a processor. For example, the functionality of the elemental imagegenerator 1528 may be implemented with an application specificintegrated circuit, in logic implemented in a processor, in logicimplemented in a specialized graphics processing unit such as the GPU1508, or in any other device.

FIG. 16 is a block diagram showing computer readable media 1600 thatstore code for rendering elemental images to be presented via a headmounted display. The computer readable media 1600 may be accessed by aprocessor 1602 over a computer bus 1604. Furthermore, the computerreadable medium 1600 may include code configured to direct the processor1602 to perform the methods described herein. In some embodiments, thecomputer readable media 1600 may be non-transitory computer readablemedia. In some examples, the computer readable media 1600 may be storagemedia.

The various software components discussed herein may be stored on one ormore computer readable media 1600, as indicated in FIG. 16. For example,a receiver module 1606 may be configured to receive an image to bepresented and a virtual distance from eyes of a viewer. For example, thevirtual image may include three dimensional content to be displayed in ahead mounted display. In some examples, the virtual distance may be adistance between a virtual surface and an eye center of the eyes of aviewer. In some examples, the receiver module 1606 may also receive eyetracking data. A renderer module 1608 may be configured to render astereo view of the image for each of the eyes at a virtual surfacelocated at the virtual distance. For example, the virtual surface may bea virtual plane. In some examples, the virtual surface may be acylindrical surface. In some examples, the virtual surface may be apiecewise linear approximation of a cylindrical surface. For example,the piecewise linear approximation of the virtual surface may include anumber of flat surfaces arranged in a cylindrical shape. In someexamples, the renderer module 1608 may configured to render the stereoviews using multi-resolution shading based on the eye tracking data. Insome examples, the renderer module 1608 may configured to render thestereo views using foveated rendering based on the eye tracking data. Amapper module 1610 may be configured to map pixels for each stereo viewfrom the virtual plane to elemental images of a per-eye display using aper-lens projection model. For example, the per-lens projection modelmay include a perspective projection of virtual image with a center ofprojection assigned to a particular individual lens. In some examples,each elemental images may be associated with a different lens of acurved lens array. In some examples, the mapper module 1610 may be apixel shader. In some examples, the mapper module 1610 may be configuredto map the pixels to the elemental images using a two ray castingoperation. In some examples, the mapper module 1610 may map the pixelsusing multi-resolution shading based on the eye tracking data. In someexamples, the mapper module 1610 may be configured to trace rays for aplurality of eye parameters based on a design of the head mounteddisplay and an eye model to generate a mapping between a screen of thehead mounted display and a retina of each of the eyes and store themapping in a look-up table. In some examples, the mapper module 1610 maybe configured to estimate an eye parameter in real-time using an eyepupil tracker and retrieve a mapping from a look-up table based on theestimated eye parameter. For example, the mapping may be used togenerate the elemental images. A pre-warper module 1612 may beconfigured to pre-warp the elemental images based on a per-lensdistortion model to compensate for a lens distortion. For example, eachof the lenses in a curved lens array may be associated with a customper-lens distortion model to compensate for a lens distortion of each ofthe lenses of the curved lens array. In some examples, the lenses of thecurved lens array may be similar and pre-warper module 1612 may beconfigured to use a similar per-lens distortion model on the lenses. Atransmitter module 1614 may be configured to send the pre-warpedelemental images to a head mounted display to be displayed.

The block diagram of FIG. 16 is not intended to indicate that thecomputer readable media 1600 is to include all of the components shownin FIG. 16. Further, the computer readable media 1600 may include anynumber of additional components not shown in FIG. 16, depending on thedetails of the specific implementation.

EXAMPLES

Example 1 is an apparatus for displaying stereo elemental images. Theapparatus includes two coupled eyepieces. Each of the two eyepiecesincludes a curved screen to display a plurality of elemental images.Each of the two eyepieces also includes a curved lens arrayconcentrically displaced in front of the curved screen to magnify theelemental images. Each of the plurality of elemental images is magnifiedby a different lens in the curved lens array.

Example 2 includes the apparatus of example 1, including or excludingoptional features. In this example, the curved lens array includes alens array pitch and a display spacing based on a target perceivedresolution, a target field of view, a target total thickness, and adisplay pixel pitch.

Example 3 includes the apparatus of any one of examples 1 to 2,including or excluding optional features. In this example, the curvedlens array includes a heterogeneous array of freeform lenses.

Example 4 includes the apparatus of any one of examples 1 to 3,including or excluding optional features. In this example, the curvedlens array includes a flat section, a cylindrically curved section, orany combination thereof.

Example 5 includes the apparatus of any one of examples 1 to 4,including or excluding optional features. In this example, the curvedlens array includes a patterned design, wherein principal planes oflenses of the patterned design are replicated along an arc of acurvature radius based on an eyeball radius, an eye relief, and a lensthickness.

Example 6 includes the apparatus of any one of examples 1 to 5,including or excluding optional features. In this example, the curvedlens array and the curved screen include a spherical curvature curved intwo dimensions to reduce off-axis aberrations.

Example 7 includes the apparatus of any one of examples 1 to 6,including or excluding optional features. In this example, the curvedscreen and the curved lens array are mechanically paired for changingthe lens array-to-display spacing while preserving concentricity.

Example 8 includes the apparatus of any one of examples 1 to 7,including or excluding optional features. In this example, both thecurved lens array and the curved screen are mechanically flexible.

Example 9 includes the apparatus of any one of examples 1 to 8,including or excluding optional features. In this example, the curvedlens array includes a planar surface that has been flexed orthermo-formed into a curved design.

Example 10 includes the apparatus of any one of examples 1 to 9,including or excluding optional features. In this example, the curvedlens array is replaceable.

Example 11 includes the apparatus of any one of examples 1 to 10,including or excluding optional features. In this example, a lens of thecurved lens array is electrically focus-tunable or dynamicallyswitchable.

Example 12 includes the apparatus of any one of examples 1 to 11,including or excluding optional features. In this example, the apparatusincludes a viewing zone with a box width based on a distance from an eyerotation center.

Example 13 includes the apparatus of any one of examples 1 to 12,including or excluding optional features. In this example, the apparatusincludes an eye position tracker to track the position of eyes of a userand a viewing zone including a box width at each pupil based on adistance from an eye rotation center or an error of margin of the eyeposition tracker.

Example 14 includes the apparatus of any one of examples 1 to 13,including or excluding optional features. In this example, the apparatusincludes an eye relief limit that is based on a shape of a viewing zoneof the apparatus.

Example 15 includes the apparatus of any one of examples 1 to 14,including or excluding optional features. In this example, the curvedscreen includes an organic light emitting diode (OLED) display.

Example 16 is a method for generating elemental images. The methodincludes receiving, via a processor, an image to be presented and avirtual distance from eyes of a viewer. The method also includesrendering, via the processor, a stereo view of the image for each of theeyes at a virtual surface located at the virtual distance. The methodfurther includes mapping, via the processor, pixels for each stereo viewfrom the virtual surface to elemental images of a per-eye display usinga per-lens projection model. The method also further includespre-warping, via the processor, the elemental images based on a per-lensdistortion model to compensate for a lens distortion. The methodincludes sending, via the processor, the pre-warped elemental images toa head mounted display to be displayed.

Example 17 includes the method of example 16, including or excludingoptional features. In this example, mapping the pixels is performedusing a pixel shader.

Example 18 includes the method of any one of examples 16 to 17,including or excluding optional features. In this example, the virtualsurface includes a plane.

Example 19 includes the method of any one of examples 16 to 18,including or excluding optional features. In this example, the virtualsurface includes a cylindrical surface or a piecewise linearapproximation of a cylindrical surface.

Example 20 includes the method of any one of examples 16 to 19,including or excluding optional features. In this example, mapping thepixels to the elemental images includes using a two ray castingoperation.

Example 21 includes the method of any one of examples 16 to 20,including or excluding optional features. In this example, the methodincludes receiving eye tracking data, wherein rendering the stereo viewsor mapping the pixels includes using multi-resolution shading.

Example 22 includes the method of any one of examples 16 to 21,including or excluding optional features. In this example, the methodincludes receiving eye tracking data, wherein rendering the stereo viewsincludes using foveated rendering.

Example 23 includes the method of any one of examples 16 to 22,including or excluding optional features. In this example, the methodincludes tracing rays for a plurality of eye parameters based on adesign of the head mounted display and an eye model to generate amapping between a screen of the head mounted display and a retina ofeach of the eyes and storing the mapping in a look-up table.

Example 24 includes the method of any one of examples 16 to 23,including or excluding optional features. In this example, the methodincludes estimating an eye parameter in real-time using an eye pupiltracker and retrieving a mapping from a look-up table based on theestimated eye parameter, wherein the mapping is used to generate theelemental images.

Example 25 includes the method of any one of examples 16 to 24,including or excluding optional features. In this example, the per-lensprojection model includes a perspective projection of virtual image witha center of projection assigned to a particular individual lens.

Example 26 is at least one computer readable medium for generatingelemental images having instructions stored therein that direct theprocessor to receive an image to be presented and a virtual distancefrom eyes of a viewer. The computer-readable medium also includesinstructions that direct the processor to render a stereo view of theimage for each of the eyes at a virtual surface located at the virtualdistance. The computer-readable medium further includes instructionsthat direct the processor to map pixels for each stereo view from thevirtual plane to elemental images of a per-eye display using a per-lensprojection model. The computer-readable medium also further includesinstructions that direct the processor to pre-warp the elemental imagesbased on a per-lens distortion model to compensate for a lensdistortion. The computer-readable medium further includes instructionsthat direct the processor to and send the pre-warped elemental images toa head mounted display to be displayed.

Example 27 includes the computer-readable medium of example 26,including or excluding optional features. In this example, thecomputer-readable medium includes instructions to map the pixels using apixel shader.

Example 28 includes the computer-readable medium of any one of examples26 to 27, including or excluding optional features. In this example, thevirtual surface includes a plane.

Example 29 includes the computer-readable medium of any one of examples26 to 28, including or excluding optional features. In this example, thevirtual surface includes a cylindrical surface or a piecewise linearapproximation of a cylindrical surface.

Example 30 includes the computer-readable medium of any one of examples26 to 29, including or excluding optional features. In this example, thecomputer-readable medium includes instructions to map the pixels to theelemental images using a two ray casting operation.

Example 31 includes the computer-readable medium of any one of examples26 to 30, including or excluding optional features. In this example, thecomputer-readable medium includes instructions to receiving eye trackingdata, wherein rendering the stereo views or mapping the pixels includesusing multi-resolution shading based on the eye tracking data.

Example 32 includes the computer-readable medium of any one of examples26 to 31, including or excluding optional features. In this example, thecomputer-readable medium includes instructions to receive eye trackingdata, wherein rendering the stereo views includes using foveatedrendering based on the eye tracking data.

Example 33 includes the computer-readable medium of any one of examples26 to 32, including or excluding optional features. In this example, thecomputer-readable medium includes instructions to trace rays for aplurality of eye parameters based on a design of the head mounteddisplay and an eye model to generate a mapping between a screen of thehead mounted display and a retina of each of the eyes and store themapping in a look-up table.

Example 34 includes the computer-readable medium of any one of examples26 to 33, including or excluding optional features. In this example, thecomputer-readable medium includes instructions to estimate an eyeparameter in real-time using an eye pupil tracker and retrieve a mappingfrom a look-up table based on the estimated eye parameter, wherein themapping is used to generate the elemental images.

Example 35 includes the computer-readable medium of any one of examples26 to 34, including or excluding optional features. In this example, theper-lens projection model includes a perspective projection of virtualimage with a center of projection assigned to a particular individuallens.

Example 36 is a system for displaying stereo elemental images. Thesystem includes two coupled eyepieces. Each of the two eyepiecesincludes a curved screen to display a plurality of elemental images.Each of the two eyepieces includes a curved lens array concentricallydisplaced in front of the curved screen to magnify the elemental images.Each of the plurality of elemental images is magnified by a differentlens in the curved lens array.

Example 37 includes the system of example 36, including or excludingoptional features. In this example, the curved lens array includes alens array pitch and a display spacing based on a target perceivedresolution, a target field of view, a target total thickness, and adisplay pixel pitch.

Example 38 includes the system of any one of examples 36 to 37,including or excluding optional features. In this example, the curvedlens array includes a heterogeneous array of freeform lenses.

Example 39 includes the system of any one of examples 36 to 38,including or excluding optional features. In this example, the curvedlens array includes a flat section, a cylindrically curved section, orany combination thereof.

Example 40 includes the system of any one of examples 36 to 39,including or excluding optional features. In this example, the curvedlens array includes a patterned design, wherein principal planes oflenses of the patterned design are replicated along an arc of acurvature radius based on an eyeball radius, an eye relief, and a lensthickness.

Example 41 includes the system of any one of examples 36 to 40,including or excluding optional features. In this example, the curvedlens array and the curved screen include a spherical curvature curved intwo dimensions to reduce off-axis aberrations.

Example 42 includes the system of any one of examples 36 to 41,including or excluding optional features. In this example, the curvedscreen and the curved lens array are mechanically paired for changingthe lens array-to-display spacing while preserving concentricity.

Example 43 includes the system of any one of examples 36 to 42,including or excluding optional features. In this example, both thecurved lens array and the curved screen are mechanically flexible.

Example 44 includes the system of any one of examples 36 to 43,including or excluding optional features. In this example, the curvedlens array includes a planar surface that has been flexed orthermo-formed into a curved design.

Example 45 includes the system of any one of examples 36 to 44,including or excluding optional features. In this example, the curvedlens array is replaceable.

Example 46 includes the system of any one of examples 36 to 45,including or excluding optional features. In this example, a lens of thecurved lens array is electrically focus-tunable or dynamicallyswitchable.

Example 47 includes the system of any one of examples 36 to 46,including or excluding optional features. In this example, the systemincludes a viewing zone with a box width based on a distance from an eyerotation center.

Example 48 includes the system of any one of examples 36 to 47,including or excluding optional features. In this example, the systemincludes an eye position tracker to track the position of eyes of a userand a viewing zone including a box width at each pupil based on adistance from an eye rotation center or an error of margin of the eyeposition tracker.

Example 49 includes the system of any one of examples 36 to 48,including or excluding optional features. In this example, the systemincludes an eye relief limit that is based on a shape of a viewing zoneof the apparatus.

Example 50 includes the system of any one of examples 36 to 49,including or excluding optional features. In this example, the curvedscreen includes an organic light emitting diode (OLED) display.

Example 51 is a system for displaying stereo elemental images. Thesystem includes two coupled eyepieces. Each of the two eyepiecesincludes means for displaying a plurality of elemental images. Each ofthe two eyepieces includes means for magnifying the elemental images.The means for magnifying the elemental images is concentricallydisplaced in front of the means for displaying a plurality of elementalimages. Each of the plurality of elemental images is magnified by adifferent lens in the means for displaying the plurality of elementalimages.

Example 52 includes the system of example 51, including or excludingoptional features. In this example, the means for magnifying theelemental images includes a lens array pitch and a display spacing basedon a target perceived resolution, a target field of view, a target totalthickness, and a display pixel pitch.

Example 53 includes the system of any one of examples 51 to 52,including or excluding optional features. In this example, the means formagnifying the elemental images includes a heterogeneous array offreeform lenses.

Example 54 includes the system of any one of examples 51 to 53,including or excluding optional features. In this example, the means formagnifying the elemental images includes a flat section, a cylindricallycurved section, or any combination thereof.

Example 55 includes the system of any one of examples 51 to 54,including or excluding optional features. In this example, the means formagnifying the elemental images includes a patterned design, whereinprincipal planes of lenses of the patterned design are replicated alongan arc of a curvature radius based on an eyeball radius, an eye relief,and a lens thickness.

Example 56 includes the system of any one of examples 51 to 55,including or excluding optional features. In this example, the means formagnifying the elemental images and the means for displaying theplurality of elemental images include a spherical curvature curved intwo dimensions to reduce off-axis aberrations.

Example 57 includes the system of any one of examples 51 to 56,including or excluding optional features. In this example, the means fordisplaying the plurality of elemental images and the means formagnifying the elemental images are mechanically paired for changing thelens array-to-display spacing while preserving concentricity.

Example 58 includes the system of any one of examples 51 to 57,including or excluding optional features. In this example, both themeans for magnifying the elemental images and the means for displayingthe plurality of elemental images are mechanically flexible.

Example 59 includes the system of any one of examples 51 to 58,including or excluding optional features. In this example, the means formagnifying the elemental images includes a planar surface that has beenflexed or thermo-formed into a curved design.

Example 60 includes the system of any one of examples 51 to 59,including or excluding optional features. In this example, the means formagnifying the elemental images is replaceable.

Example 61 includes the system of any one of examples 51 to 60,including or excluding optional features. In this example, the means formagnifying the elemental images is electrically focus-tunable ordynamically switchable.

Example 62 includes the system of any one of examples 51 to 61,including or excluding optional features. In this example, the systemincludes a viewing zone with a box width based on a distance from an eyerotation center.

Example 63 includes the system of any one of examples 51 to 62,including or excluding optional features. In this example, the systemincludes means for tracking the position of eyes of a user and a viewingzone including a box width at each pupil based on a distance from an eyerotation center or an error of margin of the means for tracking theposition of eyes.

Example 64 includes the system of any one of examples 51 to 63,including or excluding optional features. In this example, the systemincludes an eye relief limit that is based on a shape of a viewing zoneof the apparatus.

Example 65 includes the system of any one of examples 51 to 64,including or excluding optional features. In this example, the means fordisplaying the plurality of elemental images includes an organic lightemitting diode (OLED) display.

Not all components, features, structures, characteristics, etc.described and illustrated herein need be included in a particular aspector aspects. If the specification states a component, feature, structure,or characteristic “may”, “might”, “can” or “could” be included, forexample, that particular component, feature, structure, orcharacteristic is not required to be included. If the specification orclaim refers to “a” or “an” element, that does not mean there is onlyone of the element. If the specification or claims refer to “anadditional” element, that does not preclude there being more than one ofthe additional element.

It is to be noted that, although some aspects have been described inreference to particular implementations, other implementations arepossible according to some aspects. Additionally, the arrangement and/ororder of circuit elements or other features illustrated in the drawingsand/or described herein need not be arranged in the particular wayillustrated and described. Many other arrangements are possibleaccording to some aspects.

In each system shown in a figure, the elements in some cases may eachhave a same reference number or a different reference number to suggestthat the elements represented could be different and/or similar.However, an element may be flexible enough to have differentimplementations and work with some or all of the systems shown ordescribed herein. The various elements shown in the figures may be thesame or different. Which one is referred to as a first element and whichis called a second element is arbitrary.

It is to be understood that specifics in the aforementioned examples maybe used anywhere in one or more aspects. For instance, all optionalfeatures of the computing device described above may also be implementedwith respect to either of the methods or the computer-readable mediumdescribed herein. Furthermore, although flow diagrams and/or statediagrams may have been used herein to describe aspects, the techniquesare not limited to those diagrams or to corresponding descriptionsherein. For example, flow need not move through each illustrated box orstate or in exactly the same order as illustrated and described herein.

The present techniques are not restricted to the particular detailslisted herein. Indeed, those skilled in the art having the benefit ofthis disclosure will appreciate that many other variations from theforegoing description and drawings may be made within the scope of thepresent techniques. Accordingly, it is the following claims includingany amendments thereto that define the scope of the present techniques.

What is claimed is:
 1. An apparatus for displaying stereo elementalimages, comprising: two coupled eyepieces, each of the two eyepiecesincluding: a curved screen to display a plurality of elemental images;and a curved lens array concentrically displaced in front of the curvedscreen to magnify the elemental images, wherein each of the plurality ofelemental images is magnified by a different lens in the curved lensarray, wherein the curved lens array includes a dynamic or switchablelens array to vary a virtual image distance, and wherein each of theelemental images are mapped for each of a stereo view from a virtualsurface of pixels by: selecting a projection center, for each elementalimage, from a range of locations along a line extending through an eyecenter and a lens center of the different lens in the curved lens arraythat magnifies the corresponding elemental image; finding a firstprojection of each pixel of each elemental image onto a correspondingvirtual image pixel of one of a plurality of curved virtual images, thefirst projection to pass through the lens center of the different lensthat magnifies the corresponding elemental image; and finding a secondprojection of each virtual image pixel associated with each firstprojection of each pixel of each elemental image, the second projectionto pass through the projection center onto one of a plurality of virtualimage planes at different virtual distances.
 2. The apparatus of claim1, wherein the curved lens array includes a lens array pitch and adisplay spacing optimized for a target perceived resolution, a targetfield of view, and a target total thickness, given an existing displaypixel pitch.
 3. The apparatus of claim 1, wherein the curved lens arrayincludes a heterogeneous array of freeform lenses.
 4. The apparatus ofclaim 1, wherein the curved lens array includes a flat section, acylindrically curved section, or any combination thereof.
 5. Theapparatus of claim 1, wherein the curved lens array includes a patterneddesign, wherein principal planes of lenses of the patterned design arereplicated along an arc of a curvature radius based on an eyeballradius, an eye relief, and a lens thickness.
 6. The apparatus of claim1, wherein the curved lens array and the curved screen include aspherical curvature curved in two dimensions to reduce off-axisaberrations.
 7. The apparatus of claim 1, wherein the curved screen andthe curved lens array are mechanically paired to change a lensarray-to-display spacing while preserving concentricity.
 8. Theapparatus of claim 1, wherein both the curved lens array and the curvedscreen include mechanically flexible surfaces.
 9. The apparatus of claim1, wherein the curved lens array includes a planar surface that has beenflexed or thermo-formed into a curved design.
 10. The apparatus of claim1, wherein the curved lens array is replaceable.
 11. The apparatus ofclaim 1, wherein a lens of the curved lens array is electricallyfocus-tunable or dynamically switchable.
 12. The apparatus of claim 1,further including a viewing zone with a box width based on a distancefrom an eye rotation center.
 13. The apparatus of claim 1, furtherincluding an eye position tracker to track a position of an eye of auser and a viewing zone including a box width at each pupil based on adistance from an eye rotation center or an error of margin of the eyeposition tracker.
 14. The apparatus of claim 1, further including an eyerelief limit that is based on a shape of a viewing zone of theapparatus.
 15. The apparatus of claim 1, wherein the curved screenincludes an organic light emitting diode (OLED) display.
 16. A methodfor generating elemental images, comprising: obtaining, via a processor,an image to be presented and a virtual distance from eyes of a viewer;rendering, via the processor, a stereo view of the image for each of theeyes at a virtual surface located at the virtual distance; mapping, viathe processor, pixels for each stereo view from the virtual surface toelemental images of a per-eye display using a per-lens projection model,wherein the mapping of the pixels for each stereo view from the virtualsurface to the elemental images includes: selecting a projection center,for a first elemental image of the elemental images, from a range oflocations along a line extending through an eye center and a lens centerof a first lens in a lens array, the first lens associated with thefirst elemental image; finding a first projection of each pixel of thefirst elemental image onto a corresponding virtual image pixel of one ofa plurality of curved virtual images, the first projection to passthrough the lens center of the first lens that magnifies the firstelemental image; and finding a second projection of each virtual imagepixel associated with each first projection of each pixel of the firstelemental image, the second projection to pass through the projectioncenter onto one of a plurality of virtual image planes at differentvirtual distances; pre-warping, via the processor, the elemental imagesbased on a per-lens distortion model to compensate for a lensdistortion; and sending, via the processor, the pre-warped elementalimages to a head mounted display to be displayed.
 17. The method ofclaim 16, wherein the mapping of the pixels is performed using a pixelshader.
 18. The method of claim 16, wherein the virtual surface includesa plane.
 19. The method of claim 16, wherein the virtual surfaceincludes a cylindrical surface or a piecewise linear approximation of acylindrical surface.
 20. The method of claim 16, further includingreceiving eye tracking data, wherein the rendering of the stereo viewsor the mapping of the pixels includes using multi-resolution shading.21. The method of claim 16, further including receiving eye trackingdata, wherein the rendering of the stereo views includes using foveatedrendering.
 22. The method of claim 16, further including tracing raysfor a plurality of eye parameters based on a design of the head mounteddisplay and an eye model to generate a mapping between a screen of thehead mounted display and a retina of each of the eyes and storing themapping in a look-up table.
 23. At least one non-transitory computerreadable medium comprising instructions that, in response to beingexecuted on a computing device, cause the computing device to: obtain animage to be presented and a virtual distance from eyes of a viewer;render a stereo view of the image for each of the eyes at a virtualsurface located at the virtual distance; map pixels for each stereo viewfrom the virtual surface to elemental images of a per-eye display usinga per-lens projection model, wherein, to map the pixels for each stereoview, the computing device is to: select a projection center, for afirst elemental image of the elemental image, from a range of locationsalong a line extending through an eye center and a lens center of afirst lens in a lens array, the first lens associated with the firstelemental image; find a first projection of each pixel of the firstelemental image onto a corresponding virtual image pixel of one of aplurality of curved virtual images, the first projection to pass throughthe lens center of the first lens that magnifies the first elementalimage; and find a second projection of each virtual image pixelassociated with each first projection of each pixel of the firstelemental image, the second projection to pass through the projectioncenter onto one of a plurality of virtual image planes at differentvirtual distances; pre-warp the elemental images based on a per-lensdistortion model to compensate for a lens distortion; and send thepre-warped elemental images to a head mounted display to be displayed.24. The at least one non-transitory computer readable medium of claim23, wherein the instructions cause the computing device to estimate aneye parameter in real-time using an eye pupil tracker and retrieve amapping from a look-up table based on the estimated eye parameter,wherein the mapping is used to generate the elemental images.
 25. Themethod of claim 16, wherein the mapping of the pixels for each stereoview from the virtual surface to the elemental images further includesfetching a color point from a virtual image plane and setting the colorpoint as an elemental image pixel.